Supplemental material for: Efficient coordinate-descent for orthogonal matrices through Givens rotations

Definition 1. Riemannian gradient The Riemannian gradient ∇f(U) of f at point U ∈ Od is the matrix UΩ, where Ω ∈ Skew(d), Ωji = −Ωij = ∇ijf(U), 1 ≤ i < j ≤ d is the directional derivative as defined in Eq. 1 of the main text, and Ωii = 0. The norm of the Riemannian gradient ||∇f(U)|| = Tr(∇f(U)∇f(U) ) = ||Ω||fro. Definition 2. A point U∗ ∈ Od is asymptotically stable with respect to Algorithm 1 if it has neighborhood V such that all sequences generated by Algorithm 1 with starting point U0 ∈ V converge to U∗. Theorem 1. Convergence to local optimum (1) The sequence of iterates Ut of Algorithm 1 satisfies: limt→∞ ||∇f(Ut)|| = 0. This means that the accumulation points of the sequence {Ut}t=1 are critical points of f . (2) Assume the critical points of f are isolated. Let U∗ be a critical point of f . Then U∗ is a local minimum of f if and only if it is asymptotically stable with regard to the sequence generated by Algorithm 1.